The set of hyperparameter
values tried during hyperparameter tuning is
known as the search space. The definition of the range of possible values
that can be chosen, depends on the type
of hyperparameter. Some hyperparameters
required discrete values. In other words, you must select the value from a particular
set of possibilities. You can define a
search space for a discrete parameter using a choice from a list
of explicit values, which you can define
as a Python list, a range, or an arbitrary set
of comma separated values. You can also select
discrete values from any of the following
discrete distributions; qnorma, quniform, qlognormal, and qloguniform. Some hyperparameters
are continuous. In other words, you can use
any value along a scale. To define a search
space for these value, you can use any of the
following distribution types; normal, uniform,
lognormal, and loguniform. To define a search space
for hyperparameter tuning, create a dictionary with the appropriate
parameter expression for each named hyperparameter. For example, in
the search space, it can indicate
that the batch size hyperparameter can
have the value 16, 32, or 64, and the learning rate hyperparameter
can have any value from a normal distribution
with a mean of 10 and a standard
deviation of three.